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ABSTRACT 

The monograph details a process for evaluating 
programs for behaviorally disordered students as well as the results 
of that analysis. Initial chapters consider the establishment of 
criteria for well-conceptualized programs and review of a broad 
literature search on programs for this population. Di f f i cult i es are 
noted with minimal information on many components of 81 identified 
programs. A chapter on outcome summarized the 81 programs as a group 
according to quality of discussion on eight program criteria: 
philosophy, goals, population, entry, methods, exit, evaluation, and 
operations. Among findings were that exit criteria was the weakest 
element described, while methodology and operations discussions were 
the strongest. A final chapter cites questions raised by the data, 
including reasons for over-representation of mental health programs 
and programs for adolescents and the nuture of distinctions between 
programs and programming. Implications for teacher education and the 
sharing of professional information are considered. (CL) 
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CHAPTER 1 
INTRODUCTION 



Ori g1na 1 I ntentio ns 

Over the past four years, the National Needs Analysis Project staff has 
collected data from a variety of soutxes and conniuni cated at length and in depth 
with a wide range of professionals in the area of behavior disorders. Numerous 
reoccurring concerns have been expressed as part of that data collection pro- 
cess and by those professionals. Many of those concerns have been addressed 
in previous project publications. Outstanding among those concerns was one 
which was repeatedly considered important by virtually everyone with whom 
the staff talked. That issue was the description and evaluation of programs 
for behavioral ly disordered cfiildren and youth. Any attempt to seriously 
address an issue of such magnitude requires an extensive commitment of time 
and energy. Nevertheless, w;th some trepidation and a large dose of naive 
enthusiasm, the National Needs Analysis Project selected that task as one 
of the major activities for 1982-1983. 

The project staff initially conceived of an intense and rigorous effo 't 
that would involve a careful, indepth review of the full breadth of literature 
on program description and evaluation in the area of behavior disorders. A 
systematic process (see Chapter 2) was outlined to ensure such a rigorous and 
comprehensive effort. It was projected that in a year's time all relevant 
literature on programs for the behavioral ly disordered would have been con- 
sidered in depth and a docum.ent produced that discussed the types and ranges 
of programs for the behaviorally disordered as well as providing some evaluativ 
discussion concerning those programs. 
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Altered Plans 

After several months of work, it became apparent that the quality and 
breadth of the existing literature base called into question the reality of 
developing a valid discussion of programs in either a descriptive or an 
evaluative manner. If we may lapse into the informal, it is an understatement 
to say that we were discouraged and depressed. It was clear that, while the 
process we had pursued still appeared to be a good one, the literature base 
to which it was being applied was not. After numerous agonizing discussions 
between ourselves, the problem was presented to the interagency cadre of 
professionals who participate in the project's inservice phase. 

While the project staff were resigned to the possibility of abandoning 
the effort, if need be, the professionals with whom we conferred encouraged 
us that there might be value in sharing both the process and its failure with 
the field. It was felt that such an analysis of the literature base was 
instructive, raised questions that needed to be considered and might provide 
useful direction for those currently adding to the literature base; hence, 
the publication you have before you. 

Accordingly, Chapter 2 of this document discusses the literature review 
process as originally conceived and carried out. This will allow the reader 
to judge the usefulness and appropriateness of such a technique for reviewing 
the literature on the topic of programs for behaviorally disordered children 
and youth. Chapter 3 begins by summarizing the information that resulted t ru,,. 
the process and closes with a discussion of the questions raised by that summary. 
Finally, Chapter 4 discusses some considerations that professionals in the field 
of behavior disorders might address in the future. 



CHAPTER 2 
DELINEATION OF THE PROCESS 



Background 

The last (and to our knowledge, only) major program efficacy and description 
study in the field of behavior disorders was completed in 1964 by Morse, Cutler 
and Fink. At that time they estimated that the 117 programs included in their 
study represented approximately 75'.^ of the public school programs available. 
That number did not include non-public programs such as residential facilities, 
private placements, correction facilities, etc. Of the 117 programs in Lhe 
study, 54 received site visits. Those 54 programs served 519 students. For 
the purposes of rougn comparison, using that ratio, the entire sample of 117 
programs served approximately 1,123 students. Taking this extrapolation one 
step further, if that figure (1,123) represents approximately 75% of the public 
school programs in 1964, then 1,497.6 pupils or about 1,500 students, is roughly 
indicative of the number of behavior disordered students served by the public 
school s . 

By contrast, the Fou rth Annual Report to C on9iie_ss_.^n_JhLJ!]]^^ 
0fJWM^aw_94:il4^ (1982) indicates that public schools served 348,954 
seriously emotionally disturbed children in 1980-81. That is a growth rate 
of 99.4:^. In other words, statistically speaking, virtually all services 
for behaviorally disordered children and youth have come into existence since 
the time of the Morse, Cutler and Fink study. Given those data, it seemed 
appropriate to review, via the literature, the area of programming in behavior 
disorders, both in program description and efficacy. 

The initial task was to determine wtiat, exactly, constituted a program 
of service for behaviorally disordered children and youth for the purposes of 
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this endeavor. What differentiates a program from a set of ideas and/or a 
compilation of strategies? The work of McCauley (1977) appeared to provide 
the answer to that dilemma. McCauley proposed a set of program elements 
v;hich he suggests should be reflected in a wel 1 -conceptual i zed program. That 
set of elements was found to be the most complete, yet flexible, of the options 
considered and, therefore, became the criteria for the literature review, i.e., 
the presence of most of those elements was deemed necessary to consider a 
given description as one of a program. This process is more fully explained 
in an upcoming section. While the final list of program elements/criteria 
and their descriptions presented below were varied sl-^ghtly in some instances, 
they represent, primarily, the work of McCauley, 

Des^n£tj^onjD^^ 

McCauley' s assumption was adopted that there are eight essential elements 
of a well-conceptualized program. These include: 

1. fjlij QSQp 0 r Ideational Con^te^x:t. A program for behavioral ly 
disordered children and youth should have a wel 1 -articul ated 
conceptual, philosophical and/or theoretical base which includes 
a definition of disturbance, a description of population needs and 
a rationale for the program which is expressed in statements of 
belief, assumptions and/or principles. 

(^rojj^ram Goals. fM^ograni goals should include general aims and 
(; -lOSGS of the program with specific, iiRMsuiab I e objectives of 
Lhdl program (objectives for the cJi^M1ts are not. a substitute). 
Ihe goals should be consistent with the realities, that is, 
wifn thu plii losophical oersportive, the methods selected, tMo 
p(.)i)!! ! at ion served and tiu' (miv i ronmontal setting. 
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3. Population Definition . This program element includes a 
delineation of student characteristics and needs, eligibility 
issues, and the program/child match considerations. 

4. Pr ogram E ntry_. A program for behavioral ly disordered children 
and youth should: a) establish refe-ral procedures (including 
referring persons or agencies, referral priorities, specification 
of referral steps, persons in the intake orocess and data to be 
included); b) establi:,ii identification procedures (including 
methods, persons involved and steps in the sequence); and c) 
adequately address due process issues (including parental involve- 
(Ment, child rights, data access, indepenHent evaluations, negotiation 
of objectives and plans atid the right to counsel). 

5. MPt;hor! ^, Jujxiculuni_an_d_r-1 a t e r i a 1 s . In addition to establishing the 
parameters of the methods, curriculum, and materials, a program 
should describe those choices clearly, indicate their relationship 
to the program as a whole, delineate the role of the service pro- 
vider within thei,i, and organize them in such a manner that personnel 
can operate cohesively. 

6. Exi t Procedu res.. This element of a wel 1 -conceptual i zed program 
includes a delineation of a) criteria for success, b) the steps 
in the exit process, c) the persons who make exit decisions, and 
d) follow-up and tracking procedures. 



7. Evaluation. Included here are evaluation of program components, 
evaluation of child progress, and delineation of the method for 
utilization u( evaluation results for program change. 
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8. Program Opera tion. There are nuinerous related items that must be 
finalized in a coni[)lete program. These include: discussion of 
the physical facilities, 'including location and adequacy; delineation 
of program supervision and administrative responsibility; and 
consideration of financial issues, public relations efforts, 
replication possibilities, staffing issues, support personnel need 
and avai 1 abi 1 fb; , and program size. 

Ap plication of Criteria 

It was accepted from the onset that, while programs reflected in the 
literature might include these eight elements, they would not be included in 
as much detail as is implied in the above discussion. Rather, the element 
descriptions served as a guide to the range of information that might be 
present in a given source as proof of the fact that the element in question 
had been considered. Thus, the eight elements of a wel 1 -conceptua 1 i zed program 
became the criteria for determining if a given article actually represented 
a progrcm as opposed to a collection of ideas and/or strategies. The exact 
method of that process is discussed below. 

C omputerized Lit erature Se_a_rcji 

The first step in the actual review process was to obtain a computerized 
literature search of all program description and efficacy related publications. 
Tne ERIC and ECER data systems were searched for all such publications from 
1960 to the present. Search descriptors utilized were: program evalualMi), 
program effectiveness, demonstration programs, jircgi mi descriptions aiid 
alternative programs. Population descriptors ufili/od wore: omolir.n.H 
disturbance, autism and behavior' |:roblcms. That: search yielded 70 1 r.r;.)ur<:es 
for our consideration. Obviously, with sucfi broad de-.cr i i. tors , t,l,pre virrr 
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many entries not directly related to behavior disorders program description 
evaluation. 



First Level Sort 

In order to determine those entries that were most likely to yield infer- \ 
mation on programs, it was necessary to conduct a first level sort in order to 
eliminate those entries clearly ouLside of the parameters of this review. Articles 
with a primary focus on these topics were iminated: a) juvenile delinquency 
only; b) populations clearly not beha vi or,-; : iy disordered or emotional ly dis- 
turbed; c) programs or strategies for prevention of behavior disorders; d) parent 
training programs; e) programs or strategies for pre-school children; f) pre- 
service teacher education programs; and g) inservice teacher education programs. 
Of the original 703 entries, 246 were eliminatcc on this first sort.. The 
remaining 457 entries were targeted for first-hand perusal. Due to the diligent 
effort of project support personnel, U,'3 of the 457 entries were obtained for 
perusal by project staff. Thus, project staff auplic-d the process delineated 
below to 365 separate sources. 

Adopt ion of Program Def i ni t mn_arKjJ>^tj;n^^ 

The next major step in the review process was to finalize the definition 
of a program and to adopt criteria for selecting from among the remaining 365 
entries the articles to be included in the final review. These program elements, 
i.e., criteria, discussed in detail earlier are listed beiow. In brief, it 
was determined that in order to qualify as a oruMram, as opposed to a strategy 
or a technique, the following elements must :! i L^rn.ib 1 o : 

1. Philosophy or Ideational (ontt\\t: 

2 . Program Goal s 

3. Population Definition 

4. Program Entry 
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Methods, Cut^r icu 1 U;]i and Materials 
Lxit PrO(..:^^viures 
Lval ua ti on 

icdied previously, these elements were adapted from an article by McCauley 
in wiiicf] h- sets forth a listing of program elements. 
Tier rr.jci^ discussion, ir was determined that any literature entry 
:u:L-! to describe six out of the eight elements in order to be included 
■ iiiKj! sample. The reason for this limit was threefold: 

\l recogrmzed that articles (perhaps because of sheer space 
lii;, nations) do not always include descriptions of all program 
eleiuenus, even though they may be in place. Further, the 
staie-ot - the-art suggests that man> ^ehavior disorders progt^ams 
are not as 'wel 1 -conceptual i zed as they might be and thus many 
lete programs might still lack an element or two; 



[]y tfie same token, i r was the writers' goal to insure that the 
review focused on total program approaches to behavioral ly dis- 
ordered children and youth, culling out those strategies and 
techniques which, while useful, do not constitute total program 
approaches. For that reason, criteria neecied to be sufficiently 
hi-h to assure that those types of efforts were not selected; and 

The reliability procedures followed by project staff confirmed 
that six out of eight elements as a criteria seemed to 
,r ■ urately distinguish programs from strategies and still 
.il iow some flexibility for the written presentation of the 
en tr ies . 
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E§JJ ab1 1 i ty C he ckl riq_ 

Since two researchers would be reviewing articles independently, it 
was important to ascertain that the same interpretation of criteri;; was being 
applied across researchers. Thus, the staff randomly selected ten articles 
and independently -eviewed them against the program elements/criteria. In 
this subsequent comparison, the staff agrcoii o, nine out of ten articles for 
a 90. reliability. In the process of comparing, each program element was re- 
examined for eacli article or;'l additional discussion ensued concerning issues 
raised, problems encountered, etc. The staff then repeated this reliability 
checking with ten additional randomly selected av-ticles. The reliability on 
that comparison was 100.. At that poi nt the project staff felt comfor table 
to proceed independently. 

Seco nd Level Sort 

The next step involved applying the program definition and criteria to 
the 365 sources that remained following the first level sort. As discussed, 
project staff proceeded independently on this task. Each source was examined 
in detail to detennine if it met the criteria for inclusion in the review. 
Of the 365 sources, only 96 sources, representing 81 programs, were eventually 
selected as representing discussions of total program approaches for behavioral ly 
disordered children and youth. Although earlier reliability between observers was 
excellent, the project staff decided to do periodic checking on "borderline" 
or problematic entries. Thus, the staff met periodically to make joint decisions 
on sane of the sources. As was mentioned, 96 resources were eventually 
selected for inclusion in the final review. 

Serio us Q uestio ns Arise^ 

Long before the second level sort was complete, it was obvious that there 
were serious problems. In general, program elements were so poorly described 
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in the literature that the criteria v;as met by the most minimal of discussions; 
for example: a) "The children were referred by classroom teachers" as the 
only statement concerning Program Entry; b) "A program for students with 
behavior problems" as the onl> .Latement concerning Population Definition; 
or c) ^'Foi low-up indicates zndi most 01 our students have made progrur, ' as 
the only statement regarding Evaluation. It was agreed that no matter how 
minimal, if the source spoke at all to a given element it would be considered to 
have been addressed. Even given such leeway, only 81 progr^ams were identified 
out of the 365 reviewed. Clearly, the vast majority of resources in the lit- 
erature base did not include even the most minimal of a program description. 
Further, of the 81 sources eventually identified as programs, the majority 
were embarrassingly weak in their description of numerous of the program 
elements. At this point it was clear to the project staff that these 81 
programs were in no way representative of good examples of well -conceptual i zed 
programs according to the standards applied. Additionally, an unexpected result 
was what appeared to be an over-representation of adolescent programs compared 
to elementary ones and an over-representation of mental health programs compared 
to public school programs. By over-representation we are referring to the fact 
that these programs assumed a much larger percentage of the literature than 
they do of actual service delivery options. In light of all of the above, 
serious questions were raised regarding the validity of any subsequent analysis 
which described or evaluated programs or their element parts. 

As indicated in Chapter 1, it became apparent that the task, as originally 
designed, must be abandoned or revamped. The re-definition is reflected in 
the content of this document. The next chapter will suniinari^ie the data briefly 
for the purpose of sharing its overall inadequacy. 
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CHAPTER 3 
THE OUTCOME 



SuMifiia ry Data on Programs 

Even after it was determined to alter the document's original course, 
tliere v.-as Gtill discussion concetviing in what form, if any, the 81 selectcf' 
programs should be discussed. Two alternatives were considered: (1) to sum- 
marize and evaluate the 81 progran.s but only when preceded by a lengthy discussion 
of the concerns and disclaimers regarding the programs' representativeness and 
generalizability. There was concern, however, that despite disclaimers reading 
audiences might still view the programs as "model" ones, or as examples of best 
practice. Since it was clear to us that such an interpretation was clearly 
inaccurate we were hesitant to pursue that alternative; or (2) to omit all 
mention of the 81 programs other than the process used to select them and the 
final number selected. Instead, the docum.ent could focus at length on the 
reasons for not including specific details and descriptions. The concern 
here is that readers are provided with so little information that nothing is 
gained beyond the fact that an extensive literature review led to discouragement 
and an unwi 1 1 i -".gness on the authors' part to share the problems. 

Eventually, a middle course was selected. It was determined to summarize 
for the reader tne 81 programs as a group without identifying any one program 
individually. In that manner it was possible to adhere to the new focus, 
i.e., a discussion of the literature base, in general, without endorsing or 
undermining individual programs. 

To achieve this, project staff re-reviewed the 81 selected sources. On 
this review each program was evaluated on the thoroughness of the discussion 
of each of the program elements that served as criteria. Please Note: the 
rating was not based on the perceived quality of the content of the element 
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descripticn, but on the [io-xeive.! r,,iality of the diini^sjoii of ,! .jiv n t:\>m'nl. 
For example, an exemplai'y rating on [)hiloso|)hy does not mean that t'fio stalf 
perceived that the content of the philosophy was excellent, rather it means that 
the discussion of the philosophy was lh_0L0iK]ll. We a-e greatly concerned that 
the summary data may be misconstrued to mean that 20'::. of an programs have exemplar 
philosophy when, in fact, the interpretation is that 20,. oi the programs liucl 
excellent (thorough) discussions of philosophy. With that caveat, the summaries 
of tlie 31 selected programs appear in Table 1. 

TABLE 1 



Quality of Discussion by Percentagf 
of the Total Selected Programs 



Prograiii Elements 


Exempl ary_ 


Acceptabl e 


Weak 


Not Present^^ 


Phi losophy 


20'. 


26V. 


42';; 


1 2 : 


Goals 


1 2 :. 


25-:: 


53:; 


10, 


Popul ati on 


12' 


T 2 ': 


46';: 


0': 


Entry 


15; 


26:. 


42': 


1 7-. 


Methods 


28/ 


4V:: 


31';; 


0' 


Exi t 


9 : 


1 1 ;: 


28::. 


[;; ■;) 


Eval uation 


t 

' 25 ■ 


32 : 


26 :; 


1 7 ; 


Operati ons 


L " 


52';. 


17 





Thus, of the 31 programs reviewed, 20;: had thorough, i.e., exemplary, discussions 
of philosophy, 26;; had acceptable discussions, 42;:: had weak discussions and 12X 
did not address program philosophy. 

While it is probable that the reader will find most of the exemplary and 
acceptable percentages to be quite low, after reviewing the overall quality 
of the original 365 sources, project staff were surprised at how high some 
percentages were. None were lower than staff anticipated. A few observations 
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I.:.: •;: ,n^j iiioli"; M .mi .■'!(.■ mi-ilit, t.-xpCM:t. llowevfr, one coiiipniiLMi t; ol 
.'..iiMin Ival iJ;U. ii"'M i -(.iiusMir proMf.-,':. ='val ua I. i on and many |a-(iii)-ai;!S 
.,,1 ..tial.Mi!, \)roi\v(:'\'; :v- Phair solo i^;f.-asuro ot pnajraiii evaluation. 

Ma.: .„.M^ib in-d iioria.n tapo ol oaoiiipMarv and ao(apO:,-ib 1 o Idaujtaiin : V>M'a t i :)n'-, 
i-^ ,|uiia;: Iviph (. . ' da: /-'I ) and vaap/ mi s 1 ead i n(i . Prograia 0;aa-~ 
^, ; ],,., v.'OfO coiiiro i-ed al a Irirpo nuiiihoc of i nde|)fcndent variable!;, 
, . 1 1 lioa , iundinp, pol)ll> ao 1 a t i ons , e tc . Therefofe, brief 

■ t at'Oiioots on eacaa oi .evorai var-iabies or an extended dis^aisaion 
.o; any rate loctor resulted i !i hi oh iii.irks wren, in fact, the tTiader 
-;tii! f|.;P a liiiited underof^cHidino of the pcneral propran operations. 

Lxii Criteria as a asavnarM e leincnt is by far the weakest. Only 20- 
o\ the dll proprams had more than si sentence or two describinp how 
scudents e^ited tneir urograna Perhaos this is not surp-isinp in 
liplit of Idle often leveled caiticisni that behaviorally disordered 
students are prone to be locked into programs and have great di f- 
riculty iiiovincj back into the mai ns traain. 

Sixty-nine percent of all programs had exemplary or acceptable 
discussions of methodology. This fact, not a surprising one, has 
generated much discussion among project staff and project participants 
In the opinion of the project staff and as supported by the percentage 
it appears that methods are the most useful part of the literature 
base. The literature base, both the 81 selected programs and the 
remaining unselected sources, strongly reflects Eigqramming strategies 
rather than programs. While the common perception is that one may 
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(,i:rn Lo thu 1 i toi'd Lurr to Icar-ii about [jfocji'diiis for boliav i oral ly 
d i '".ordert.'Li LtiildrX'ti .uid youtli, it is iiiuro a reflection of reality 
lliat one liia.y turn to tlie lit(M\iture for prog rammi ng. Perhaps, (jiven 
the C4i!ficult.y of j-.roiji-ain replication from one environment to the 
next, tlie latter is more useful. More discussion on this phenomenon 



lable 2 simriar i .:es tliu per'C'.ai t<u!es a little diiderently. What 
is reflected is a collapsing of the percentages into two categories 
(1) exemplary and acceptable, and (2) weak or not present. In 
very gross terms this reflects a dichotomy between acceptable 
versus not acceptable. Given that perspective, on four of the 
eignt elements, over 50'': of the selected programs had very poor 
or no discussions. Another two (Population and Evaluation) 
approach that. Of the two items. Methods and Program Operations, 
that appear to be wel 1 -di scussed in the 81 programs, the latter 
is misleading. (Please see Number 2 for discussion.) Only the 
program element of Methods could be expected to be thoroughly 
discussed in most sources. Even so, 69% is not overwhelmingly 
impressive. Even if the literature is focusing primarily on 
p rogramming as discussed in Number 4 above, it cannot be said 
to be doing it unusually well. 
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IMii 1 oso|.)!iy 
(ioa! s 

i 'opu 1 a t: i on 
Lntr\y 

Methods 
Exi t. 

Evaluation 
Opet\"it ions 



QuaTTf:y oT'lJ i scuss fon' by' Percon Lc 



[)''\ 

2i: 



72 



Weak ot;^ No_t_ 



L 
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Where does all this load us? As the old joke goes, "There's some good 
news and some bad news." The good news is that a comprehensive look has now 
been taken at the current literature base in Line field of behavior disorders 
in terms of program description and evaluation. The bad news is that there 
are now :,iany more questions and no answers. The final chapter will consider 
where to go from here. 
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111)1 Sl IONS AND CONSIDI l-;Al IDN 



iju-: ". I. i 0111= ■•'L^l. 

What Liiei) did wo irMfn I roin >,uch d i>f'r,r,/'l cd Llic literatAiro and micIi 
"I iiuiiiKis?" In deidiUon to liow to overcome despair, the Droject staff loarned 
t,, ,,-;!, M!|M^ f. ioir--, . Vlliil';' literally huruh'eds niioht come to iiiind, the Tocir; ti(>re 
1;, on a t(nv thai, were foutKl to be iiarticularly Lrouoliny. 

i. Wliy is the literature not reflective of practice? We know, and 
trust tiiat readers also know, that there are in existence more 
l)ro()rani5 in the lield serving children and youth who are handi- 
capped by their behavior than even the 365 which were encountered 
before the application of the eight program elements. Why are 
these programs not reflected in the literature? Do they evclv 
too quickly to ever be committed to paper? Is there no one 
connected with the program who has the time and/or inclination 
to share it in written form? Are t^^e programs just not fully 
conceptualized, i.e., having most if not all of the progran, 
elements in place? Are most services to behavior disordered 
students a collection of pronramiv.ing strategies which have 
never been conceived as part of a total program approach? Given 
the difficulty of replicating programs in other environments, is 
ii of concern that total program descriptions are not well reflected 
in the literature? 

2. Why do mental health programs appear to be over-represented in 
the literature? The vast majority of behavioral ly disordered 
students are served in public school classes, yet the majority 
of programs reflected in the literature are mental health and/or 
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of [)iil),ic M.hf>t)l [M'f)<]r\iiir., tor l.lw' Ix^lidv iora 1 ly '1 i sor(.l^M''■Mt? 
/\^i^ ()ut)li<. school unv i r'or)!ii(Mits lotc\lly uncoriduc i vo to the 
[)rppcuvi uon ot liutoricUs \ ■.)v |)ut)l icatii)n ? ilow can the inclusiuii 
of }iubl ic scliool i^ro^jraifis in l.Uo 1 i t(M\) turri be (jifoctod? 

I,'i)y ^y.. tf^V : ; ' ; ) ^ ' ' . Ml t / . f I 1 M M"' 0 ! ) 0 T f i 0 M a tO 1 y Vep TO S 0 tO' t 

u) ttiL^ litoraturoV Public school pt^ogranis usually have a few 
more eleinenvary programs or, in sonte cases, have approximately 
(?(]ual numbers of elementary and adolescent programs. Yet, 
()rograms for the behavior disordered adolescent dominate tiie 
literature (in our selected sample, slightly over 2 to 1 ) . Do 
job descriptions vary for persons involved in adolescent programs 
allowing them the opportunity to share programs in a written 
ashion? Does the field feel that they have elementary programs 
under control" but are still searching at the adoler.cent level? 
Are more adolescent programs funded by grants which then have a 
greater tendency to have a written expression of the program? 
Finally, is there any relationship between the representation of 
mental health programs and adolescent programs? Perhaps the 
adolescent representation is merely a reflection of the mental 
health representation. At the time of data collection, the staff 
did not tally the number of adolescent programs there were des- 
criptions of mental health programs versus the number that were 
descriptions of public school programs. However, staff recollections 
coincide: that the vast majority of adolescent programs described 
were mental health programs. If the literature over-represents 
mental health programs and mental health programs publish more 
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ab.Mit, tJM.'ir >.i..lo .Mil. i.ro.ir.nir,, Wwu Mir ddo I (.■■,<;rii t. ovr-r 
ri-|ii-i;s(Mit.a t. iiiii won Id nn\ Iw as \n.i : r 1 i ml . 

4. Uhdl cxaLLly <ir(,. t.hr' d i L i ik t i oos be tv.-oni |)ru<jfaiii'. .irui |.r'0.jranii:i i n-!? 

Was our conct-pL ol |)ro(irains unrealistic.^ lliinecossary ? Of f - Lar,- i ? 

Is prtxiraniming surficient; t,o inoiM. st.udcMil. iioods? Do |,r'.M|ramiMi lui 
niitions consti lute nrofirains ? 

11. is obvious that (1) tlio (luestions liiat could have been raised lollowimj 
ttiis past year's work are infinite. We selected some of those that were most 
concerning to us; and (2) the options for more detailed dTSCUssions tiiat arise 
from those questions are equally infinite. Again, we have selected those 
that are particularly troubling to us based on our own work and the work of 
those persons who have been the advisors, participants and data sharers fo,- 
this project. Below, the reader will find three brief discussions that rai^e 
the concerns we have experienced as a result of the attempt at review and 
analysis of programs for the behavioral ly disordered. 

Re 1 i a ne e on the L jJ^eL^Jiyi 'g ^o*" Guidanc e 

What are the implications of this endeavor and its results for: (1) 
using the literature base in teacher training at all levels, but especially at 
the graduate level; and (2) using the literature as a starting place in 
program development? 

It is not only customary, it goes virtually without question, that, 
graduate students will be "sent to" the literature base to learn about and 
synthesize a variety of issues in the area of behavior disorders. Based upon 
the data we saw, while one might support that approach for programming strat- 
egies, it would be difficult for these authors to continue supporting it as 
a vehicle for teaching about program descriptions or program efficacy. If, 
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1),.. tJiL' uiiiy ••■JO, I, .M.lci'l.h in!:wi;;iM-n !h,,i ,i .;.,nlu,it,' '.lud-Ht > iv-'. .il.'-.^' 



prOlllMMr, , I I will ii ' il i f I iui 1 t i i' 



1 i,r 'i i '.oriliM ', , .1'. ^ I i ■ ■ 1 'I , ' <' T' 



!,. ^.r-v il,. 1 u.j'.T .1 Ir tln'i::- ■ inr:- -Ml lan-ilM;;! 01. I.I. M,', ' 01 I,. 

havuu.iMy .! i .ou-dc--^ .Uioron arul yoii'l,. i-v i-id, tirid .Ld./. d p,., i n ! 

o pro.|i\\:!i, iiiiL ' i i.Uo l.novi I cdijo ni !)ro(ii\)i:i', . 

i.irly, IJio u',(; of the lUoraturo 1)0 .• :, .i I'nnit, ui dopof t tj. not, 
only ior do:,c. i:-.iVL' but .j!so for olfioa.y in'onn.iU-oo ho ,iiiluo,d iti 
progrdin devo; 1 opir.on is collod into question. In fact, wo learned tduit l.w, 
of our colleagues in state department of education roles went to the literaturo 
m an effort to select a general set or sets of" or.oru.tTon for- the developmient 
and sophistication of behavior disordered programs in their state. They loft 
the process convinced that it offered little guidance for the philosophical 
issues in program development much less th.^ implementation issues. If not 
in the literature, where? What guidance is available to persons whose job 
it is to direct and influence program development on a large scale? 

CoiK ppj-u^ij Joramew^or k v s . Comp i 1 e d_J]g,c!]JliiL4ej_ 

One of the most troubling questions that has arisen concerns the issue 
of programs vs. programming. Have we, the authors, or we, the field, lost 
perspective on the relative value of programs vs. programming? Could it be 
more useful for the literature to reflect programming rather than programs? 
If so, what is the appropriate way to share programs? If programming is the 
more valuable content, where is the guidance for combining assorted strategies 
into a unified whole? 
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IvJ Ui i nl- w-: afu Luo b iasril 



s ; . lM^"' i i i.:'ra Lur:> 

t; i on ro^ia i ns : dot^ ^ 



u^vf I ' o'^ .til 1 h" 
I ' ad i'^^prosont ac tua 



) f'i^M t: 1 y ro!:>o I vou i n 



1 i [) Stjrv i CO the' I 



practice in ravor ot roLUMn^j on ijoo^jramiiii rio dt^s[nt- uoe . 
■jivon to pronraius, IT tfio litorature 'lo-. not -Pflo.t or-aoti^o Lbor^ tfu- ot[.,r 
major coricorns ^^^ised apply, (hv: Mirthor powit lo ^onsiJio- is that wh il p tho 
litorature is Kiore rotlucrivo o^ proqra-nmi no . it io sti"! not unusual ]y strono 
in that area, either (see page 14, nuinbor o;, 

F!ow_bs_ Information Shared? 

If, as this literature reviev/ appears to indicate, tie literature base 
is not reflecting programs in behavior disorders arid is only doing a fair job 
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al)i 1 1 ty ■/^ r^r I i -a ^.t; 



pro 



: . , Mir ' (; i 'P (h^ t i . ; n iiart 'M 
\ • -rii.j 1 ) ■ :ii sh.ir ! Mp at^' 

..M' ! o!is pv - i t^.- * li'^ f ie Ivl ' - 
■ : ; to-. Im i puos . W i thou t 

p-ufjfMinn: i na st'Vitepies, roi) 1 i CvU i on 



IS not po'.i.ibIc. Without roplK.)tion, spr^'i:-. run tfi- risk of becorninq pro< 
vincicil, narrow and/or ''>^o- invonti np thr wh^ol/ In the lon^j run, replication 
ai'l: efficiency and efficiency allows ^ jr- offers tc bo focused on proactive 
plannincj of prograins or oroyraninrwig stratecjie^. . 

Ao point of interest, these authors are curious about the quality and 
coniprehensiveness of the literature base in other areas of special education, 
not for the reasons as some may suspect, that misery loves company (although it 
sometimes does). Rather, it would be useful in our^ search for answers to know 
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;ln: tli i ■ docui'^erit ::-in^it servo a sL'iniUiUS 



' -^.icna is 



a i ter- it i v^-''- , There has obvious ly been no such at:t-r,)p 
;ihj-': outhor' iifwe been ove; whcliiied by nieeelv posit in- tfu- 



and shared, we thinh, by 



At nedi-i i' ee as suiiioti n :ieid by t'lese diithO!-: 
our colleagues rha;, the literature base is an everorowing reoository ir^e 
;ie riiat is available ir. assist, and guide us as wc strivo' to 
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orovKie services lo children and youth who are handncapped by theii- behavior. 
■ air- -^e-ience nas not suio. rro.: this a^ -o.nntu>n , at least as U relates so 
. .J.,scriotion and e- - . Id ■■r. a,r.ious ihah: ^l) the as-Horaion 

; fc; the ouestions raised lulate Oiouoht in ordm- that the 
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